Processing container images and identifiers using optical character recognition and geolocation

ABSTRACT

Embodiments include a system configured to process location information for objects in a site comprising an imaging device configured to take a picture of an object, the picture containing a unique identifier of the object; a global positioning system (GPS) component associated with the imaging device and configured to tag the image of the object with GPS location information of the object to generate a tagged image; a communications interface configured to transmit the tagged image to a server computer remote from the imaging device over an Internet Protocol (IP) network; and a processor of the server configured to perform Optical Character Recognition (OCR) on the picture and to create an indicator code corresponding to the identifier of the object, wherein the processor is further configured to create a processed result containing the indicator code and the location to locate the object within the site.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation application of U.S. patentapplication Ser. No. 14/017,078 filed Sep. 3, 2013, which is acontinuation-in-part of U.S. patent application Ser. No. 13/422,442entitled “Method and Apparatus Processing Container Images and/orIdentifying Codes For Front End Loaders or Container Handlers ServicingRail Cars,” filed Mar. 16, 2012, which is a Continuation of U.S. patentapplication Ser. No. 12/262,130 (Now U.S. Pat. No. 8,146,813), filed onOct. 30 2008, which in turn claims the benefit of Provisional PatentApplication No. 60/983,888 filed Oct. 30, 2007, which are allincorporated herein by reference.

TECHNICAL FIELD

Embodiments relate generally to container management systems forshipping and storage yards, and more specifically to imaging systemsthat process container identifiers using optical character recognitionand geo-location technology.

BACKGROUND

Container terminals that service ships and trains, and store containersfor end-user delivery rely on containers being transported in and out ofthe port via truck traffic. These containers are usually placed on acontainer chassis to transport the containers to a give destination thatusually requires travel roads, or rail if the container and chassis areplaced on a rail car for transport inland. When the container arrives atthe destination, the container may be delivered without the chassis ormoved to a different chassis for various other reasons, such asmaintenance to a specific chassis.

Container chassis may have many different owners. Some are privatelyowned, others may be owned in a “pool” by several terminals in aspecific area, and some may be owned by a specific company, such as atransportation company. It is very important for the person or companythat is currently using the chassis to track the location of the chassisso the owner can maintain state and government registrations of thechassis and also allocate the chassis to the appropriate container loaddepending on its location. Since container chassis often travel hundredsor even thousands of miles to a specific destination, tracking chassiscan be a very challenging task.

Chassis are typically tracked with an alphanumeric number that is uniqueto each particular chassis. A typical identifier format is “ABCD 123456”(where ABCD is any unique combination of letters and 123456 is anyunique combination of numbers. The identifier may be placed or writtenon the sides, front, and back of the chassis. Terminals have a means oftracking a chassis as it arrives in the terminal gate, such as by usingan Automated Gate System (AGS) to capture digital camera images of thechassis number on the front, sides, and back of the chassis as ittravels through the gate. Once the chassis is inside a containerterminal or storage area, the chassis is either allocated to a newcontainer or is stored. Finding the chassis can often be very difficultas they are often stored in various arrangements (e.g., horizontal orvertical stacking) to minimize space.

One drawback associated with present chassis storage methods is thatthere is still a requirement to track and maintain a list of all chassisstored at a site so they can be matched to the right container when thechassis leaves the terminal facility. This is also important forvaluation of chassis assets at a specific site and availability for newshipments. Current methods to track the chassis inventory requireemployees of the terminal site to manually climb in and around thesevertically and horizontally stored chassis to record the chassis numbersand the location of the chassis within the terminal. As chassisinventory at a given site can vary greatly depending on the volume ofcontainers and the number of chassis being moved at a given time, thismethod of recording chassis can be very labor-intensive andtime-consuming. Maintaining the chassis inventory list can require teamsof people to work full time writing or entering in chassis numbers intoan inventory database. Furthermore, such work can be dangerous, as itmay often occur in dark areas with extreme wet or cold conditions thatcan create hazardous and slippery conditions to personnel as they movearound the chassis and storage site to record chassis identifierinformation.

The subject matter discussed in the background section should not beassumed to be prior art merely as a result of its mention in thebackground section. Similarly, a problem mentioned in the backgroundsection or associated with the subject matter of the background sectionshould not be assumed to have been previously recognized in the priorart. The subject matter in the background section merely representsdifferent approaches, which in and of themselves may also be inventions.

SUMMARY OF EMBODIMENTS

Embodiments are directed to systems and methods for managing inventoryof objects, such as containers in a container yard or other largeobjects that are associated with unique identifiers for location withina site. In an embodiment, an apparatus comprises an imaging deviceconfigured to take a picture of an object, the picture containing analphanumeric identifier of the object, a location identifying barcode ora global positioning system (GPS) component associated with the imagingdevice and configured to tag the image of the object with GPS locationinformation of the object to generate a tagged image, a communicationsinterface configured to transmit the tagged image to a server computerremote from the imaging device over an Internet Protocol (IP) network,and a processor of the server configured to perform Optical CharacterRecognition (OCR) on the picture and to create an indicator codecorresponding to the identifier of the object, wherein the processor isfurther configured to create a processed result containing the indicatorcode and the location to locate the object within a site. The object maycomprise a container of at least twenty feet long and a chassisconfigured to carry at least one container, and wherein the tagged imagecomprises metadata including the identifier and information of thecontainer or of handling equipment of the container. The imaging devicemay comprise a handheld smartphone having a camera and atelecommunications interface. In an embodiment, the GPS componentcomprises a circuit resident in the smartphone. The imaging device maybe a camera mounted on a mobile vehicle configured to drive around thesite.

The processor may be configured to receive said tagged image through aweb site. In an embodiment, the tagged image is displayed and processedin a web page displaying location information of the object in the site.The web page comprises a plurality of display areas showing the locationof the object relative to a map of the site and a description of thelocation relative to defined coordinates of the site. The apparatus mayfurther include an audit process providing display area configured toallow a user to correct an identifier processed by the OCR in the eventof an error character recognition.

The site may be a container storage and transport terminal, and whereinthe imaging device is mounted on at least one of: a top-handler, arubber tire gantry crane. The imaging device may alternatively bemounted at a gate location of the site for imaging container transportvehicles as they enter or exit the site.

Any of the embodiments described herein may be used alone or togetherwith one another in any combination. The one or more implementationsencompassed within this specification may also include embodiments thatare only partially mentioned or alluded to or are not mentioned oralluded to at all in this brief summary or in the abstract. Althoughvarious embodiments may have been motivated by various deficiencies withthe prior art, which may be discussed or alluded to in one or moreplaces in the specification, the embodiments do not necessarily addressany of these deficiencies. In other words, different embodiments mayaddress different deficiencies that may be discussed in thespecification. Some embodiments may only partially address somedeficiencies or just one deficiency that may be discussed in thespecification, and some embodiments may not address any of thesedeficiencies.

BRIEF DESCRIPTION OF THE DRAWINGS

In the following drawings like reference numbers are used to refer tolike elements. Although the following figures depict various examples,the one or more implementations are not limited to the examples depictedin the figures.

FIG. 1 shows an example of a container yard and/or a shipyard containerhandling system including a container imaging and processing systemunder embodiments.

FIG. 2 shows an example of a front end loader, in particular a tophandler, coupled to multiple cameras configured to create images ofvarious faces of the container.

FIG. 3 shows some details of the optical characteristic system and/orthe container handler, including at least one light and at least onecamera.

FIG. 4 shows a refinement of some aspects of FIG. 1, where the serverinterface further includes the second processor communicatively coupledto a second means for communicating to receive the member of the opticalgroup from a container handler and/or from an optical characteristicsystem coupled to the container handler.

FIG. 5 illustrates a composition of any of the processors, under someembodiments.

FIG. 6 shows some details of the sensor module and the machine state ofFIG. 3, under some embodiments.

FIG. 7 shows an example of some of the details of the operationalschedule of FIG. 1, under some embodiments.

FIG. 8 shows a flowchart of the program system of FIG. 5, under someembodiments.

FIGS. 9A and 9B show some container images of the container code asdisplayed on various faces of the container, under some embodiments.

FIG. 9C shows a container code estimate derived from the containerimages, under some embodiments.

FIG. 10 shows an example of a handheld device generating a tagged imageof an indicator on a surface of an object, under an embodiment.

FIG. 11 illustrates some examples of the indicators for containers andchassis.

FIG. 12 shows the tagged image being used for an insurance estimateabout the object, under an embodiment.

FIG. 13 shows how the insurance estimate and/or the image used for theestimate may also be included in the processed result.

FIG. 14a shows a communication device configured to access the handhelddevice to send the tagged image to the processor to create the revenue,under an embodiment.

FIG. 14b shows an example web site that may be configured to provide thetagged image to the processor to create the revenue.

FIG. 14c shows a handheld device that may include any combination of theGlobal Positioning System (GPS) receiver, under an embodiment.

FIG. 15a shows some example components the handheld device of FIG. 1 mayuse and/or include to at least partly create the location, under someembodiments.

FIG. 15b shows some examples of an access portal that may be used by thecommunication device to send the tagged image.

FIG. 16a shows some examples of the components of the processor, the website and/or the database, under an embodiment.

FIG. 16b shows some examples of the components of the processor, the website and/or the database, under an alternative embodiment.

FIG. 17 is a flowchart showing an example of the details of a programsystem, under an embodiment.

FIG. 18 shows some details of the storage of the chassis that may bereported by the database and recorded by the handheld device.

FIG. 19 illustrates a set of barcodes representing unique storagelocations for a container location processing system, under anembodiment.

FIG. 20 illustrates a sample web page displaying container locationinformation, under some embodiments.

DETAILED DESCRIPTION

Embodiments relate to operating at least one camera to create an imageof a container by an apparatus on a container handler or included in acontainer handler, or other apparatus communicating with the containerhandler apparatus, and/or estimating the container's code while thecontainer handler is servicing a transport element. The transportelement may be a rail car, truck, ship, or other appropriate vehiclethat moves or transports a container.

A first embodiment includes an optical characteristic system configuredto couple to a container handler configured to transfer a containerto/from a transport element to create at least one member of the opticalgroup. The optical group consists of a container image of at least partof at least one face of the container, an optical characteristic derivedfrom the container image, a container code estimate of a container codeon the face, a transport element identifier estimate of a transportelement identifier on the transport element and a container chassisidentifier estimate for a container chassis that may hold the containerduring at least part of the transfer.

A second embodiment includes a server interface configured to receivethe at least one member of the optical group from at least one of thecontainer handler to create another of the members of the optical groupand/or a manifest for at least the transport element and/or aconfirmation of the manifest and/or an insurance record for thecontainer. The container handler may be a front end loader, a reachstacker, or a gantry crane. The front end loader may be a top handler ora side picker. The gantry crane may be a quay crane and/or a rail gantrycrane. Alternatively, the gantry crane may be a rubber tire gantrycrane.

A third embodiment includes an optical characteristic system configuredto couple to a container handler configured to transfer a container tocreate at least one member of the optical group, where the containerhandler is a front end loader.

A fourth embodiment includes a container handler configured to transferat least one container to/from a transport element and configured toreport at least one member of the optical group.

A fifth embodiment includes a server computer configured to receive atleast one member of the optical group from at least one containerhandler to create another of the members of the optical group and/or amanifest for at least the transport element and/or a confirmation of themanifest and/or an insurance record for the container. The servercomputer (also referred to as a “server”) may be embodied asprocessor-based computer server that is kept in a computer server roomwithin or remote from a transport or shipping facility (such as in arail kiosk or other remote location), or it may be any appropriatelyconfigured server computer located and operated in the vicinity of, orremotely from the transport or shipping facility in which the containersare handled.

A sixth embodiment operates a server configured to receive at least onemember of the optical group from at least one container handlerconfigured to transfer a container and/or operates the server interfaceto receive from the container handler and/or the optical characteristicsystem for the container transferred to/from the transport elementand/or operates the container inventory management system to receivefrom the server and/or the server interface. The optical characteristicderived from the container image may include an optical enhancement ofthe container image. It may also be preferred that the opticalcharacteristic further includes a compression of the container imageand/or the optical enhancement.

The insurance record may be created using at least one of the containerimages for each of at least two members of the face group of thecontainer. The face group consists of a top face, a bottom face, a frontface, a back face, a left face and a right face, each of the container.

In certain embodiments, the container handler and/or the opticalcharacteristic system may include a processor communicatively coupled toat least one camera configured to create the container image of thecontainer, possibly for transfer to/from the transport element. Theprocessor may also be communicatively coupled to a means forcommunicating to send at least one member of the optical group.

In certain embodiments, the server interface and/or the server mayinclude a second processor communicatively coupled to a second means forcommunicating to receive the member of the optical group from acontainer handler and/or from an optical characteristic system coupledto the container handler, possibly with the container handlertransferring the container to/from a transport element. The secondprocessor may create at least one items discussed above for the serverinterface and/or the server.

The container handler and/or the optical characteristic system may beconfigured to wirelessly report at least one of the optical groupmembers. The server interface and/or the server may be configured towirelessly receive the report.

The container handler and/or the optical characteristic system mayfurther include a sensor module to create a position estimate that maybe used to direct the camera to create the container image.

FIG. 1 shows an example of a container yard and/or a shipyard containerhandling system including one or more embodiments of the in operationstoring and/or transferring containers 10 with a container code 12 andpossibly using at least one transport element 4. Various embodiments ofthe invention may involve a container handler 2 transferring containersto and/or from a transport element 4 and reporting a container codeestimate 54 and/or an optical characteristic 58 and/or a container image52 of the container handled by the container handler as well as atransport element identifier estimate 47 of a transport elementidentifier 5 on the transport element and a container chassis identifierestimate 49 for a container chassis 38 that may hold the containerduring at least part of the transfer. As used herein the relationship“to and/or from” will be referred to as “to/from”.

A first embodiment includes an optical characteristic system 50configured to couple to a container handler 2 configured to transfer acontainer 10 to/from a transport element 4 to create at least one memberof an optical group. The optical group consists of a container image 52of at least part of at least one face of the container, an opticalcharacteristic 58 derived from the container image and an container codeestimate 54 of a container code 12 on the face of the container as wellas the transport element identifier estimate 47 of the transport elementidentifier 5 on the transport element and the container chassisidentifier estimate 49 of the Chassis identifier 39 as marked on thecontainer chassis 38 that may hold the container during at least part ofthe transfer.

A second embodiment includes a server interface 80 configured to receivethe at least one member of the optical group from at least one containerhandler 2 to create another of the members of the optical group and/or amanifest 40 for at least the transport element 4 and/or a confirmation42 of the manifest and/or an insurance record 46 for the container 10.

The container handler 2 may be a front end loader, a reach stacker 28,or a gantry crane. The front end loader may be a top handler 24 or aside picker 26. The gantry crane may be a quay crane 20 and/or a railgantry crane 20. Alternatively, the gantry crane may be a rubber tiregantry crane 22.

The third embodiment may also include an optical characteristic system50 configured to couple to a container handler 2 configured to transfera container 10 to create at least one member of the optical group, wherethe container handler is a front end loader, for example, a top handler24 or a side picker 26. As used herein a top handler may move acontainer in a forklift fashion and side pickers, which may also bereferred to as side handlers, may operate similarly to top handlers butare frequently used for lifting at least partly empty containers fromthe side to speed the movement of such containers.

A fourth embodiment includes the container handler 2 configured totransfer at least one container 10 to/from a transport element 4 andconfigured to report at least one member of the optical group, forexample, the container image 52, the optical characteristic 58 and/orthe container code estimate 54 as well as the transport elementidentifier estimate 47 of the transport element identifier 5 on thetransport element and the container chassis identifier estimate 49 forthe identification of chassis identifier 39 as marked on containerchassis 38 that may hold the container during at least part of thetransfer.

A fifth embodiment includes a server 32 configured to receive at leastone member of the optical group from at least one container handler 2 tocreate another of the members of the optical group and/or a manifest 40for at least the transport element 4 and/or a confirmation 42 of themanifest and/or an insurance record 46 for the container 10.

The sixth embodiment includes a container inventory management system 48configured to receive from the server interface 80 and/or the server 32at least one of the following: the container code estimate 54, themanifest 40, the confirmation 42 and/or the insurance record 46 as wellas the transport element identifier estimate 47 of the transport elementidentifier 5 on the transport element and the container chassisidentifier estimate 49 for the identification of chassis identifier 39as marked on container chassis 38 that may hold the container during atleast part of the transfer.

The seventh embodiment operates a server 32 configured to receive atleast one member of the optical group from at least one containerhandler 2 configured to transfer a container 10 and/or operate theserver interface 80 to receive from the container handler 2 and/or theoptical characteristic system 50 for the container transferred to/fromthe transport element 4 and/or operate the container inventorymanagement system 48 to receive from the server and/or the serverinterface.

In various embodiments, a container handler 2 and/or an opticalcharacteristic system 50 may be communicatively coupled 62 with theserver interface 80 and/or the server 32.

It may be preferred that the container 10 be at least twenty feet long,though other lengths 181 and sizes are also possible. By way of example,the size of the container may be ten feet, twenty feet, twenty fourfeet, thirty three feet, forty five feet or fifty three feet.

The optical characteristic 58 derived from the container image 52 mayinclude an optical enhancement of the container image. The opticalenhancement may include the removal of glare, color correction, edgeenhancement, contrast adjustment and any image filter as found in asoftware tool for image processing of digital photos and/or digitalvideos. It may also be preferred that the optical characteristic furtherincludes a compression of the container image and/or the opticalenhancement. The compression may comply with an international standard,such as a version of JPEG or MPEG.

The container handler 2 and/or the optical characteristic system 50 mayinclude a processor 14 communicatively coupled to at least one camera 6configured to create the container image 52 of the container 10,possibly for transfer to/from the transport element. The processor mayalso be communicatively coupled to a means for communicating 106 to sendat least one member of the optical group.

The processor 14 may receive the container image 52 as a raw version tocreate a filtered version, a compressed version, and/or an enhancedversion of the image as the optical characteristic 58. The filteredversion may remove or attenuate certain frequency bands of the rawversion. The compressed version may require smaller storage capacitythan the raw version; and the enhanced version may enhance contrastand/or strengthen edges detected in the raw version.

The server interface 80 and/or the server 32 may include a secondprocessor 60 to receive via the communicative coupling 62 the member ofthe optical group from the container handler 2 and/or from the opticalcharacteristic system 50 coupled to the container handler. The containerhandler may be transferring the container to/from a transport element 4.The second processor may create at least one items discussed above forthe server interface and/or the server.

As shown in FIG. 1, some embodiments of the invention may operate acontainer handler 2 loading 60 or unloading 16 the transport element 4and reporting the container estimate 54 and/or optical characteristic 58and/or a container image 52 of a container 10 handled by the containerhandler.

A transport element 4 may be stopped in a container terminal servicingat least one dock configured to load 60 and/or unload 62 a ship 64and/or in a rail yard for transferring containers between at least onerail line and at least one trucking route configured for draymen trucks34 and/or at least one warehouse (not shown) and/or at least one factory(not shown). Sometimes a container may be transferred to a transportelement from a ship 64, to a transport element from a stack 48, to atransport element from a drayman (or “dreyman”), or to a transportelement from a UTR truck 36 and/or a container chassis 38.Alternatively, the container may be transferred from a transport elementto a stack, from a transport element to a vessel, or from a transportelement to a drayman. A rail head 30 will refer to a site where atransport element may be stopped and a container may be transferred ontoor off of the transport element or rail car.

Some embodiments may manage a container inventory system 48 based uponreceiving 64 the container code estimate 54 and/or the opticalcharacteristic 58 and/or container image 52 for a container 10 beingtransferred by a transport element 4 in contact with a container yard,which may be a shipyard.

Some embodiment methods may include optically recognizing containercodes 12 of containers 10 in transfer operations involving draymentrucks 34 and stacks 48 of containers in particular by front endloaders. As used herein, a drayman vehicle is a land transport vehiclefrequently comprising a truck and at least one trailer, which cantransport containers within a container terminal and over public roadsand highways.

Various embodiments of the invention may operate in a container storageand/or transfer yard, which may service container ships, railroads,overland trucks, factories and/or warehouses supporting the automatedrecognizing of container codes 12 displayed on various sides of thecontainers 10 being stored and/or transferred. At least one processor14, 124 and/or 60 may initiate an operational process 76 by an opticalcharacteristic system 50 mounted on a container handler 2 to create anoperational result 78, select the operational process based upon anoperational schedule 130 and/or communicate 62 with at least one of theoptical characteristic systems to receive at least one container image52 of a container 10 being handled by the container handler to at leastpartly create a container code estimate 54 for transmission to acontainer inventory management system 48.

FIG. 2 shows an example of a front end loader (FEL), in particular a tophandler 24, coupled to multiple cameras configured to create images ofvarious faces of the container 10.

If an insurance record 46 is processed, it may be created using at leastone of the container images 52 for each of at least two members of theface group of the container. The face group consists of a top face 90, abottom face 96, a front face 92, a back face, a side face 94, which maybe a left face and a right face, each of the container 10, as shown inFIG. 2.

FIG. 3 shows some details of the optical characteristic system 50 and/orthe container handler 2 of FIG. 1, including at least one light 100 andat least one camera 102, both may preferably be coupled to the processor14 to receive at least one container image 52. The processor may becoupled with a sensor module 104 to create a machine state 102 used todetermine the position estimate 56 and communicatively coupled 108 witha means for communicating 106 to report the container image 52 and/orthe optical characteristic 58 (as shown in FIG. 1) and/or the containercode estimate 54. The position estimate may be used to direct the cameraand/or the lights to improve the quality of the container image.

Returning to FIG. 2, the optical characteristic system 50 and/or thecontainer handler 2 may include at least one camera 102 based upon theposition estimate 56 of a container 10 to create the container image 52.The container image may be used to create a container code estimate ofthe container code 12 on a face of the container and/or be used as afront side image of the front face 92 of the container or to imagelonger containers (e.g., 40 ft and greater). A second camera may bemounted near the top of the front end loader to create a secondcontainer image of the top face 90 of the container. A third camera maybe mounted to focus a container image where a twenty foot container'scontainer code 12 is found. A third camera may be mounted near thebottom of the front end loader to create a third container image of thebottom face 96 of the container. And a fourth camera may be positionednear the cab of the front end loader for capturing the container as itis moved for auditing purposes. A fifth camera may be added to capture adrayman truck chassis ID 49 and/or transport element chassis identifier5.

FIG. 4 shows a refinement of some aspects of FIG. 1, where the serverinterface 80 and/or the server 32 further include the second processor60 may be communicatively coupled 132 to a second means forcommunicating 134 to receive via the communicative coupling 62 themember of the optical group from a container handler 2 and/or from anoptical characteristic system 50 coupled to the container handler,possibly with the container handler transferring the container 10to/from a transport element 4. The second processor may create at leastone items discussed above received by the server interface and/or theserver. The second processor may use an optical character recognitionsystem 136 to create the container code estimate 54 from the opticalcharacteristic 56 and/or from the container image 52 received from thecontainer handler and/or the optical characteristic system.

The means for communicating 106 and/or the second means forcommunicating 134 may use a network protocol that complies with at leastone member of the group consisting of: a wireless network protocolstandard and a wireline network protocol, a time divisional multipleaccess protocol, a spread spectrum protocol, a packet transfer protocol,a frame transfer protocol, an error correction coding protocol and anencryption protocol. If a wireline network protocol is used, it mayinclude at least one of the following: a Synchronous Serial Interfaceprotocol, an Ethernet protocol, a Serial Peripheral Interface protocol,an RS-232 protocol, and Inter-IC protocol (sometimes abbreviated asI2C), a Universal Serial Bus (USB) protocol, a Controller Area Network(CAN) protocol, a firewire protocol, which may include implementationsof a version of the IEEE 1394 protocol, an RS-485 protocol and/or anRS-422 protocol.

The container handler 2 and/or the optical characteristic system 50 maybe configured to wirelessly report 62 at least one of the optical groupmembers. The server interface 80 and/or the server 32 may be configuredto wirelessly receive the report.

The means for communicating 76 and/or the second means for communicating134 and/or the wireless network access point 128 may include a radiofrequency tag terminal and/or a radio frequency transmitter and receivercompliant with at least one wireless signaling convention that mayimplement at least one of a Time Division Multiple Access (TDMA) scheme,a Frequency Division Multiple Access (FDMA) scheme, and/or a spreadspectrum scheme.

As shown in FIG. 5, at least one of the processors 14 and/or 60 and/or124 may include at least one instance of at least one of a neuralnetwork 140, a finite state machine 142, an inferential engine 146, anda computer 150 accessibly coupled 152 to a computer readable memory 154including a program system 170 and/or an installation package 156configured to affect the program system. In some embodiments, theinstallation package may create the program system and/or modify anexisting program system to implement various combinations of programsteps embodying instructions that implement one or more embodiments. Theprogram system and/or the installation package may be provided throughaccess with a download server 160, which may be provided across an opennetwork such as the Internet and/or through a private network as such avirtual private network or through the use of an access protocolpossibly using a user name and a password.

As used herein, a finite state machine 132 receives at least one input,maintains and updates at least one state and generates at least oneoutput based upon the value of at least one of the inputs and/or thevalue of at least one of the states.

As used herein, an inferential engine 134 maintains a collection ofinference rules and a fact database and responds to queries andassertions by invoking rules and accessing the fact database. Examplesof inferential engines include fuzzy logic controllers and constraintbased decision engines used to determine paths through networks basedupon the network constraints, such as the current location of parked andmoving vehicles and available storage locations for containers.

As used herein, a computer 150 includes at least one data processor andat least one instruction processor instructed by the program system 170,where each of the data processors is instructed by at least one of theinstruction processors.

As shown in FIGS. 3 and 6, the container handler 2 and/or the opticalcharacteristic system 50 may further include a sensor module 104 tocreate a position estimate 56 that may be used to direct the camera 102to create the container image 52. The camera may preferably be directedin coordination with the light 100.

FIG. 6 shows some further details of the sensor module 104 and themachine state 102 of the container handler 2 of FIG. 3. A presencesensor 202 may respond to the presence of a container 10 to create asensed container present 204. A stack height sensor 206 may create aSensed stack height 208. A Size sensor 210 may respond to the containerbeing handled to create a Container size estimate 212. A Twistlocksensor 214 may create a Twistlock sensed state 216. A Spreader sensor218 may create a Spreader sensed state 220. A Landing sensor 222 maycreate a Sensed landing state 224. And/or a Hoist sensor 226 may createa sensed hoist height 228. Any or all of the sensed states 62, 66, 70,74, 78, 82, 86 and/or 90 may be included in the machine state. Thesensor module may further communicate with a handler interface 200 tocreate the machine state.

The handler interface 200 may vary for different container handlers 2.For example when the container handler is a gantry crane such as railgantry crane 20 or an RTG crane 22, the container handler may include ameans to sense the crane spreader interface signals (landed, locked,container width), and may further, possibly separately couple to sensorson the crane hoist or trolley drum for determining the trolley and hoistposition of the crane, for instance by using a tachometer, proximitysensor, optical sensor, encoder, or laser beam. Other embodiments of thehandler interface may include a coupling to a Programmable LogicController (PLC) to capture this data. Also, the handler interface mayinclude a wireline network interface to at least one of the sensors ofthe container handler. As used herein, a wireline network interface mayimplement an interface to at least one of the wireline network protocolsmentioned herein. Embodiments are configured to operate with an RTG(rubber tire gantry crane), and RMG (rail mounted gantry crane), orsimilar crane or hoisting equipment.

Another example, when the container handler 2 is a side picker 26, a tophandler 24 or a reach stacker 28, the handler interface 200 may includea wireline network interface to at least one of the sensors of thecontainer handler. Other sensors may be accessible to the handlerinterface through separate wireline network interfaces and/or wirelinenetwork couplings.

A third example, when the container handler 2 is a UTR truck 36 or acontainer chassis 38, the handler interface 200 may include a wirelinenetwork interface to at least one, and possibly all the accessed sensorsof the container handler. Alternatively, more than one wireline networkinterfaces and/or wireline network couplings may be used.

The handler interface 200 may further receive any or all of thefollowing information that may be forwarded to the container inventorymanagement system 48: the location of the container 10, a sensedoperator identity of the operator operating the container handler 2, acontainer radio frequency tag, a container weight, a container damageestimate, an indication of the container handler moving in a reversemotion, a frequent stops count, a fuel level estimate, a compassreading, a collision state, a wind speed estimate, a vehicle speed, andan estimate of the state of a vehicle braking system.

The handler interface 200 may include a radio transceiver providing aradio frequency tag interface capable of locating the container handler2 and/or identifying the container 10 and/or its container code 12.

The handler interface 200 may include another radio transceiver using aGlobal Positioning System (GPS) and/or a Differential Global PositionSystem (DGPS) to determine the location of the container 10. The handlerinterface 200 may include an interface to a short range and/or low powerradar that may provide a position estimate 56 of the container 10.

FIG. 7 shows an example of some of the details of the operationalschedule 130 of FIG. 1, including multiple operational events 232, eachincluding a start time 234, a list of identified optical characteristicsystems 236, an operational process, and its operational result. Theoperational process may be a member of the group consisting of apower-up process 240, a diagnostic process 242, a normal process 244 anda power-down process 246.

Some of the following figures show flowcharts of at least one embodimentof at least one of the methods of the invention, which may includearrows signifying a flow of control, and sometimes data, supportingvarious implementations.

The boxes denote steps or program steps of at least one of theinvention's methods and may further denote at least one dominantresponse in a neural network 140, and/or at least one state transitionof the finite state machine 146, and/or at least one inferential link inthe inferential engine 142, and/or a program step, or operation, orprogram thread, executing upon the computer 150.

Each of these steps may at least partly support the operation to beperformed as part of a means for an operation or use. Other circuitrysuch as network interfaces, radio transmitters, radio receivers,specialized encoders and/or decoders, sensors, memory management and soon may also be involved in performing the operation further providingthe means for the operation.

The operation of starting in a flowchart is denoted by a rounded boxwith the word “Start” in it and may refer to at least one of thefollowing: entering a subroutine or a macro instruction sequence in thecomputer 150, and/or of directing a state transition of the finite statemachine 146, possibly pushing of a return state, and/or entering adeeper node of the inferential engine 142 and/or stimulating a list ofneurons in the neural network 140.

The operation of termination in a flowchart is denoted by a rounded boxwith the word “Exit” in it and may refer to completion of thoseoperations, which may result in at least one of the following: a returnto dormancy of the firing of the neurons in the neural network 140,and/or traversal to a higher node in the graph of the fact databaseand/or the rules collection in the inferential engine 142, and/orpossibly return to a previously pushed state in the finite state machine146, and/or in a subroutine return in the computer 150.

FIG. 8 shows a flowchart of the program system 170 of FIG. 5 asincluding at least one of the following program steps: Program step 172supports operating the sensor module 104 to create the position estimate56 of the container 10 as shown in FIG. 3. Program step 174 supportscontrolling the camera 6 based upon the position estimate to create thecontainer image 52. Program step 176 supports operating the means forcommunicating 106 to send the report of the member of the optical group,in particular the container image 52, the optical characteristic 104,the container code estimate 54 as well as the transport elementidentifier estimate 47 of the transport element identifier 5 on thetransport element and the bomb cart or container chassis identifierestimate 49 for the container chassis 38 that may hold the containerduring at least part of the transfer. Program step 178 supportsoperating the second means for communicating 134 to receive the reportof the member of the optical group as shown in FIG. 4. Program step 180supports creating the optical characteristic based upon the containerimage. Program step 182 supports creating the container code estimatefrom the container image and/or the optical characteristic. Program step184 supports creating the manifest 40 for the transport element 4,preferably based upon the container code estimate as generated by theserver 32 in some embodiments of the invention. Program step 186supports creating the confirmation 42 of the manifest 40 from thecontainer code estimate derived from the container image sent by thecontainer handler 2 and/or the optical characteristic system 50. Programstep 188 supports creating the insurance record 46 for the container,preferably through archiving two or more container images of the facesof the container as shown in FIG. 2. Program step 190 supports creatingthe transport element identifier estimate 47 of the transport elementidentifier 5 of the transport element 4 as shown in FIG. 1. And programstep 192 supports creating the container chassis identifier estimate 49of the container chassis identifier 39 of the container chassis 38holding the container 10 for at least part of the transfer.

FIGS. 9A and 9B show some example container images of the container code12 as displayed on various faces of the container 10. And FIG. 9C showsa container code estimate 54 derived from one of these container imagesand/or an optical characteristic 56 as discussed previously. Transportelement identifiers 5 and/or container chassis identifiers 39 may beimplemented in a fashion similar to container codes and/or may beimplemented using Radio Frequency Identification (RFID) technology.

Optical Character Recognition and Asset Location Determination with aHandheld Device

Embodiments include a system that applies and processes OpticalCharacter Recognition (OCR) of tagged images of a container and/or acontainer chassis to create an indicator code. The tagged images aregenerated by a handheld device and include at least one image of anindicator on the side of the object (container and/or chassis) and alocation. A processed result of the indicator code and the location arethen used by a management system to track the location of the object andpossibly its status, such as the temperature in a refrigeratedcontainer.

Embodiments of the invention include methods and apparatus that aid inmanaging an object. The object 8 may include a container 20 at leasttwenty feet in length and/or a chassis 24 configured to carry at leastone of the containers. A handheld device may be used to generate atagged image of an indicator on the side of the object. The tagged imagemay include an image of the indicator and a location associated with theobject. A processor receives the tagged image and responds by performingOptical Character Recognition (OCR) on the image to create an indicatorcode. The indicator code and the location may be used to update adatabase regarding the object.

Referring to the Figures, FIG. 10 shows a system that may include thefollowing: The handheld device 101 generates a tagged image 300 of anindicator 360 on the side of the object 81, such as a chassis indicator260 or a container indicator 221. The tagged image 300 may include animage 320 of the indicator 360 and a location 340 associated with theobject. A processor 1000 receives 1020 the tagged image and responds byperforming Optical Character Recognition (OCR) on the image to create anindicator code 1100. The indicator code and the location may be used asa processed result 1200 to update a database 2000 about the object 81,shown here as the object track 2080, that may include the image 320, itslocation 340, the temperature 1100 as a temperature indicator 290 of thetemperature inside a refrigerated container 280. This database may beused to generate a report 2200 about the object 81 and its location 340,which may further include the internal temperature 290 for refrigerationcontainers 280, insurance 2100 related images and reports. Theembodiments may determine that the OCR results are inaccurate and maytrigger an audit 1300 of the image 320 to create an indicator auditestimate 1320.

A processed result 1200 may include the indicator code 1100 and thelocation 340 to help locate and identify the object 81. The processedresult may also include the image 320, the temperature indication 1100,a time stamp, which may have been embedded in the tagged image 300 aswell as possibly an operator identifier of the handheld device 101.

FIG. 11 shows examples of the indicator that includes a containerindicator 221 on the side of the container 201 as well as a chassisidentifier 260 of the chassis 241 and a license plate 270 of thechassis. In many situations, the container indicator 221 may be found onthe front side, the backside (as shown in this Figure), the left side,the right side (as shown in FIG. 1), the top and the bottom of thecontainer 201. The license plate 270 is often only found on the backsideof the chassis as shown in this Figure. The chassis indicator is usuallyfound only on the backside (as shown in this Figure), the left side andthe right side (as shown in FIG. 10).

FIG. 12 shows the tagged image 300, in particular, the image 320 may beused for an insurance estimate about the object 81, shown here as adamaged container 201 that can be identified by the container indicator221.

FIG. 13 shows the processed result 1200 may include the insuranceestimate 2100, indicator code 1100, and/or the image 320 used for theestimate.

Various embodiments may implement the authenticated access 1020differently. The processor 1000 may be configured to enableauthenticated access for receipt of the tagged image 300 as shown inFIG. 10. FIG. 14a shows a communication device 1500 may be configured toaccess 1540 the handheld device 101 to send the tagged image 300 to theprocessor 1000. The communication device 1500 may use an access portal1520 to access the handheld device 101. FIG. 14b shows a web site 1600that may be used to provide the tagged image 300 to the processor 1000.In any of these embodiments, the tagged image 300 may be created.

FIG. 14c shows the handheld device may include any combination of theGlobal Positioning System (GPS) receiver 141, a Differential GPS (DGPS)receiver 1400, a barcode scanner 1420 and/or a Radio FrequencyIdentification (RF-ID) tag 1460, to at least partly create the location340. In certain embodiments, the barcode scanner may use the imagingdevice 120 of FIG. 10 to acquire a version of the location. This may beaid in locating chassis 241 and/or containers 201 that may be stacked inpositions that are not readily determined by GPS coordinates, as will beshown in certain examples below.

FIG. 15a shows some examples of communication components that may beincluded in the handheld device 101 or of the communication device 1500.These examples include, but are not limited to, a cellular phone 1600, acellular base station 1520, a Local Area Network (LAN) client, a LANrouter, a Wireless LAN (WLAN) client 1680, a WLAN access point 1700, aBluetooth client 1720 and/or a Bluetooth host 1740.

FIG. 15b shows some examples of an access portal 1500 that may be usedby the communication device 1500 to access 1540 the tagged image 300.The access portal may be compatible with a version of at least one of aUniversal Serial Bus (USB) protocol 1800, a Firewire protocol 1820, anda SATA protocol 1840.

FIG. 16a shows some examples of the components of the processor 1000,the web site 1600 and/or the database 2000. The processor, the web siteand/or the database may include at least one instance of at least onemember of an implementation group consisting of members of a FiniteState Machine (FSM) 3100, an Inferential Engine (Inf Eng) 3120, a neuralnetwork 3140 and a computer 3160 instructed 3180 by a program system3200 residing in at least one memory 3220, with at least one of themembers contributing to at least partly create and/or use the processedresult.

FIG. 16b shows some examples of the database 2000, a computer readablememory 2100, a disk drive 2120, and/or a server 2140, that may beconfigured to generate at least part of the program system 3200 in theprocessor 1000. The database, the computer readable memory, the diskdrive and/or the server may further include an installation package 2160configured to instruct the computer 3160 to install at least part of theprogram system in the processor. As used herein, the installationpackage may include any combination of source code, compiled modulespossibly implemented as linkable libraries, and compressed versions ofthe program system components. These installation packages and/or thedatabase, the computer readable memory, the disk drive and/or the servermay deliver the installation package and/or the program system 3200 toat least partly contribute to the revenue 3000, the second revenue 3020and/or the third revenue 3040 of FIG. 10.

As used herein, a finite state machine 3100 may receive at least oneinput, maintain at least one state and generate at least one outputbased upon the value of at least one of the inputs and/or at least oneof the states. As used herein, an inferential engine 3120 maintains alist of inference rules and a list of facts, to which hypotheses may bepresented to determine the consistency or inconsistency of thehypothesis to the facts based upon the rules of inference. As usedherein, a neural network maintains a list of neural states and asynaptic network connecting those neural states, which may be presentedstimulus to trigger changes in those neural states based upon thestimulus as transferred through the synaptic network. As used herein, acomputer 3160 includes at least one instruction processor and at leastone data processor with each of the data processors instructed by atleast one of the instruction processor based upon the program system inaccord with the various embodiments of this invention, which include butare not limited to the processor 1000, the web site 1600, the database2000, the computer readable memory 2100, the disk drive 2120 and/or theserver 2140. Note that in some embodiments the server 2140 may supportat least part of the web site 1600.

FIG. 17 is a flowchart of some details of the program system 3200instructing the processor 1000, under an embodiment. These flowchartsshow some method embodiments, which may include arrows signifying a flowof control and/or state transitions as well as sometimes position data,supporting various implementations. These may include a programoperation, or program thread, executing upon the computer 1040 or statesof the finite state machine 1020. Each of these program steps may atleast partly support the operation to be performed. The operation ofstarting 321 a flowchart refers to entering a subroutine or amacroinstruction sequence in the computer or of a possibly initial stateor condition of the finite state machine. The operation of terminationin a flowchart refers to completion of those operations, which mayresult in a subroutine return in the computer or possibly return thefinite state machine to a previous condition or state. A rounded boxwith the word “Exit” in it denotes the operation of terminating aflowchart.

FIG. 17 shows an example of the details of the program system 3200 ofFIGS. 16A and 16B. The program system may include at least one of thefollowing program steps:

Program step 3500 supports configuring the web site 1600, theauthenticated access 1020, and/or the processor 1000, for the processorto receive the tagged image 300. Program step 3520 supports operatingthe web site 1600 to receive the tagged image 300. Program step 3540supports performing the OCR on the image 320 to create the indicatorcode 1100. Program step 3560 supports insurance reporting at least oneinsurance image included in the tagged image 300 to contribute tocreating an insurance estimate 2100 of the object 81. Program step 3580supports auditing 1300 the tagged image 300 to further create theindicator audit estimate 1320 in response to an inaccuracy determinationof the indicator code 1100 of the image 320. Program step 3600 supportsupdating a database 2000 of the object 81 in response to the processedresult 1200 to create an update of the database. Program step 3620supports operating the database 2000 to create the report 2200 of theobject 81 based upon the indicator code 1100 and/or the location 340.

FIG. 18 shows some details of the storage of the chassis 241 that may bereported by the database 2000 and recorded by the handheld device 101.The chassis may be stacked vertically or near-vertically 261 b, orstacked horizontally 261 a. In both situations, there is a limitedability for any form of GPS readings to clearly designate the locationsof the chassis in either stack, and the operator 60 of the handhelddevice 101 may use a bar code scanner 1420 to read-off locations in astack, or to locate the stack in a storage yard or transfer facility.

Embodiments of a handheld imaging system may be used in a cameraautomation system using OCR technology that allows operators of thesystem to carry a digital camera with a GPS receiver (or similargeo-location system) on it, which they use to take pictures of thechassis number, or other identifier of a container or transport element.These cameras may use a technique known as geotagging to mark the GPScoordinates on the image so that the digital image of the chassis numberis paired to a GPS position within the container yard or other site. Forsites that have specific racks or locations of chassis, a uniqueidentifier such as a bar code or number on a sheet of paper, can beimaged at the start of the row so the chassis are paired to a specificrow within a site. These images can then be uploaded to a server, whichallow for the container number to be read by an OCR system, and thendisplayed to the user or other operator.

In an embodiment, the handheld imaging system allows site personnel towalk the areas where chassis are stacked with a supplied digital camerathat is able to embed a GPS position into the image to create a taggedimage, such as tagged image 300 in FIG. 12. A “geotagging” camera with aGPS receiver can be used to embed GPS coordinates into the images thatare taken. A camera can also be a cellular phone with a camera builtinto it. If a cellular phone is used, the image data can be transmittedto the server directly from the cellular phone. For sites that storechassis in specific rows or locations and require this level of locationdetail, the procedure can be modified slightly. For example, at thestart of the row or area to be imaged, the user will take a picture of abarcode (or other unique ID) that is used to indicate the specific row.The barcode or unique ID is on either a sheet of paper (which has abarcode assigned to each row on the page) or affixed to a location inthe area/row of containers to be imaged. FIG. 19 shows an example sheet192 containing a barcode 192 that could be used for identify containersin rows. Each barcode on the sheet 192 represents a unique locationwhere chassis (or other objects) are stored within the container yard orsite. This allows specific chassis to be located to specific rows orareas of the site.

Using this system, an entire site can be processed to provide aninventory of the chassis located at the site. For example, when theimaging is done for the row, the camera operator can close the row bytaking a picture of an “end of row” barcode/identifier. This process iscontinued until all of the containers are imaged in each specific row.When geotagging or barcoding is used, this technique can also be used toidentify other information at the start of an imaging session, which mayinclude user name, row, lot, tier (height) information related to thechassis/container numbers captured. In an embodiment, the image istagged with certain metadata in a defined format. Such metadata mayinclude the following data fields, among others: camera manufacturerdata, image orientation and resolution, software version, date/time,exposure and imaging data, flash configuration, pixel configuration, GPSlocation coordinates, and other appropriate data. Such metadata may beprovided in any appropriate file format, such as an .EXIF data.

Once the means of locating the chassis has been started (either with aunique barcode/ID Number and/or geotagging); the chassis number isrecorded via a digital camera, and may be stored in the camera as animage 320, such as shown in FIG. 12. This eliminates the need tomanually enter the chassis number into a handheld device or write itonto a pad, which requires further data entry later. As the image istaken, the GPS position of the chassis is recorded and embedded into theimage by the camera with GPS ability. This technique of imaging thechassis can be used not only for stacked chassis but also for anychassis or container in the terminal. Once processed or stored, theimage may be OCR imaged so that a representation of the chassis numberor identifier may be stamped on the image for easy viewing or furtherprocessing. Any appropriate camera or digital imaging device that hasthe requisite imaging capability, storage and processing power, and GPSfeatures may be used. Additionally, the camera may include otherfeatures, such as a zoom feature which allows for capturing chassisnumbers from a distance and allows the operator of the camera to have toget too close to the chassis stacks which is a safety hazard. Similarly,a flash or fixed LED lamp can be used for night illumination of thechassis numbers in darkness and low light condition.

When all chassis are imaged at the site, the images can be downloadedfrom the camera memory (e.g., flash card on the camera) and sent to theserver, which may be installed at the terminal site or at a remotelocation that is accessed via an Internet connection. These images canbe downloaded via a USB cable, a Wi-Fi or Bluetooth connection builtinto the camera, a cellular broadband connection, or any otherappropriate transmission means.

In an embodiment, the image is tagged with certain information toproduce a geotagged image. The following data can be used to tag theimage: an image of the chassis number, the GPS Position Data, the imagetime and date information; a unique identifier (barcode, unique ID, orfiducial) that identifies a specific area the chassis is stored in, auser name for the imaging process (stored in a barcode or unique ID)that is imaged at the start of the imaging session, location information(e.g., lot, row tier information that is imaged at the start of theimaging session, imaging conditions, and any other camera specific orsite specific information.

When the server processes the images for OCR, the following imageprocessing functions may be applied: the image is rotated, shifted orscaled if required, the contrast and white balance are adjusted on theimage to enhance the OCR result, or any other appropriate imageprocessing. The OCR engine then accesses the image to extract thechassis number. The OCR result may be overlaid on the original image incontrasting (e.g., orange) text and/or it may be sent to the server forindexing, storage or further processing. When the server has finishedapplying OCR processing to the image, the result may published to awebsite with the geographic position (from the geotagged or uniquebarcode data) on a map interface (e.g., Google maps or equivalent). Thedata can also be sent directly to another remote database or server fortracking of the chassis asset.

FIG. 20 illustrates a web page displaying data map location of a chassislocation of a chassis imaged at a site, under an embodiment. The webpage image 195 includes several image locations including a map region196 in which red “pushpins” or other indicators on the map represent thelocations of the chassis, an OCR data region 198 with a table that showsthe location information, and in which each row 197 in the tablerepresents a chassis and is numbered to the corresponding number on thered pushpin in the map image. FIG. 20 is intended to provide anillustration of one possible configuration of a web page showing numberand location information for geotagged container and/or chassis images.Other configurations are also possible, such as the type of map overlay,the format of the table, the type of data illustrated, and so on.

In an embodiment, the OCR result for the chassis may be furtherprocessed by the server, and/or sent to another system as needed. Incertain cases, the imaging process, or the OCR process may not returnclear or comprehensible images. In an embodiment, chassis numbers thatcannot be resolved by the OCR process can be corrected by a camera orimage audit system. This system allows the user to view a close up ofthe image and correct the image by manually entering the chassis numberas seen in the image. The system may provide at least two ways a usercan manually audit or correct an image. Either correction through a webbrowser and an Internet connection or an application-based audit client,though others are also possible. In a web-based audit system, for achassis image with a chassis number that was not read correctly by theOCR system, the user can enter the corrected number into a text box thatis provided through the interface below. The system can be configured toflag an incorrect or suspected number and prompt the user for review andcorrection. The user then enters corrected OCR result in the appropriatebox or location. The application based audit client window can providethe same functionality for correcting images that have no OCR results ora wrong OCR result of a container chassis, but instead of a web-basedinterface, a separate client, server, or device application interface isprovided.

In an embodiment, the handheld camera system may be operated by a userwalking, cycling, or otherwise moving around a site manually takingstill or video pictures with a digital camera. If a video system isemployed separate imaging processing may be used to convert the videoframes to stills and isolate the frame or frames that representsnapshots of containers or other images being processed. In anembodiment, the digital camera may be embodied in a smartphone thatincludes camera and GPS functionality, along with the ability to runapplications and access the Internet through a web browser interface.Such a device may be embodied as a phone, tablet, mini-tablet,mini-notebook, PDA or other appropriate handheld device that issufficiently portable for use by an operator moving around a site. Thephone may also be configured or have an installed application that canread bar codes or other visual codes that may encode the identifier.Such a device may also be used at certain pre-defined locations, such asgates, hubs, intersections within a site to routinely image, process andtransmit identifier information to the server.

Top-Handler/RTG OCR

In one embodiment, the camera system may be installed on tophandlers orfront end loaders for capturing containers as they are put or pulled atthe bombcart, chassis, or railcar height to stack or chassis. In thisembodiment, the image capture platform is a tophandler (a forklift typemachine, or a rubber tire/rail gantry crane (a crane on rail or rubberwheels for moving containers). The image data is sent to the server overthe Internet (or Intranet if hosted locally), from this mounted camera.Using such a tophandler OCR system, inventory can be updated as follows:the top-handler OCR process performs a container pull fromUTR/Chassis/Bombcart/Drayman to stack, the driver starts the Tophandler.Cameras installed on the tophandler image the side of the container onthe UTR/Chassis/Bombcart/Drayman, GPS installed on the mast provideslocation information and a PTIU installed on the equipment capturesspreader height and twistlock status. The PTIU stands for ‘PositionTracking Interface Unit’ and is a custom designed controller fordetermining terminal equipment state such as (but not limited to) heightof container, width of container, and twistlock status, and may beimplemented as a microcontroller or similar processing device mounted ona piece of transport equipment, such as a crane or container handler,but that may also be located remotely from the equipment. When a UTRbombcart, chassis, or drayman drives underneath the tophandler/RMG/RTG,the system images the side of the container when the twistlocks go toLAND and LOCK status and the container is at the correct height suchthat the container number is in the correct field of view of the camerato image. The PTIU installed on the Tophandler determines the rightcontainer height to image and to only image the container when thetwistlocks are LOCKED. The container must be locked and lifted to image,and it is imaged at the bombcart/railcar/chassis height. When thecontainer is placed in a stack of containers, the container height andcell locations are recorded by the PTIU. This image data is sent to theserver system and OCR is performed on the captured images. If the imageis unreadable, it is sent to the audit client for correction andcontainer number identification by an operator or by an automatedprocess. Once the image data is OCR processed and confirmed, the finalcontainer placement location and OCR result is updated in the systemdatabase. The system can then send a message to the terminal operator orother entity with location of container. The UTR/Chassis/Bomb CartUnique ID number may also be identified by a camera OCR/Containeridentification system at the time of container pull event to associatethe container being lost from that UTR/Chassis/Bomb Cart,

In the case of a container pull from the stack to aUTR/Chassis/Bombcart/Drayman, the process may proceed as follows: thetophandler/RTG operator pulls a requested container from stack location,the PTIU senses container pull event and determines the row, bay, celltier location of the container; based on row, bay, cell, tier, thesystem accesses the database to locate container in stack and retrievesthe container number from the database. As the container is placed on awaiting Chassis/Bombcart/Drayman from a Tophandler/RTG, the container isimaged on the side of the container once again as a double verification.The TOS inventory system is updated with container placement (onUTR/Chassis/Bombcart/Drayman from stack in yard). The UTR/Chassis/BombCart Unique ID number may also be identified with an Camera/OCR systemat the time of container put to the UTR/Chassis/Bomb Cart event toassociate the container being associated with to that UTR/Chassis/BombCart,

For a container stack-to-stack reposition operation (i.e., moving acontainer from a different location in a stack to another location, theprocess may proceed as follows: the Tophandler/RMG/RTG operator pullscontainer from stack location, the PTIU senses container pull event anddetermines the row, bay, cell tier of the container pulled. Based onrow, bay, cell tier, the process uses the database to locate containerin stack and retrieves the container number from the database. Thedriver of Tophandler/RTG/RMG then puts container in a new stack position(it is not put on a UTR or Drayman truck so cameras are not used in thiscase). The PTIU records put location of container, and the databaseupdates the location of container and sends message of the new containerlocation. In this case, no OCR may be performed it is based purely oncontainer put and pull location as determined by the GPS and PTIU.

In the case of a container pull from a UTR/Chassis/Bombcart/Drayman to arail car, the process may proceed as follows: the tophandler driverpulls a container from a UTR/Chassis/Bombcart/Drayman. When a UTRbombcart, chassis, or drayman drives underneath the Tophandler/RTG/RMG,the system images the side of the container when the twistlocks LAND andLOCK. The UTR/Chassis/Bomb Cart Unique ID number may also be identifiedwith a camera and OCR system at the time of container pull event toassociate the container being lost from that UTR/Chassis/Bomb Cart,

The PTIU installed on the Tophandler determines the right containerheight to image such that the container number is in the field of viewof the camera for OCR recognition. The container must be locked andlifted to image and is imaged at the bombcart/railcar/chassis height(typically 11-20 ft). When the container is placed on a railcar, GPS ontophandler/RTG/RMG sets this as a put to rail message. The tier (heightof the container on the rail car is recorded. Railcars typically storecontainers up to two high). Note that rules may need to be enforced toensure that the train is parked in the same location every time. Theimage data is sent to the server and OCR processed. If the image isunreadable, it is sent to the audit client. Once the image data is OCRprocessed and confirmed, the final container placement location is setas put to rail in the database, and a message may be sent indicating thelocation of the container on rail. The railcar may be identifiable via aradio tag or unique identification number which will also allow for thecontainer to be located to a specific rail road car on a train track.

In the case of a container pull from railcar to aUTR/Chassis/Bombcart/Drayman, the process may proceed as follows: thetophandler/RTG/RMG driver pulls a container from a railcar. When thecontainer is placed on a UTR/Chassis/Bombcart/Drayman, a put todrayman/UTR message is created. The container is imaged at the UTRheight and container number is read. This image data is sent to theserver and OCR processed. If the image is unreadable, it is sent to theaudit client. Once the image data is OCR processed and confirmed thesystem sends a message with location of container (on UTR, in yard). TheUTR/Chassis/Bomb Cart Unique ID number may also be identified with anCamera/OCR system at the time of container put to the UTR/Chassis/BombCart event to associate the container being put to/associated with tothat UTR/Chassis/Bomb Cart,

The geotagging/OCR process may also be used in conjunction withcontainer transport using RTG, RMG, or straddle carrier equipment. Inthis embodiment, a camera system is installed on cranes to capturecontainers as they are pulled on and off of bombcarts and chassis andplaced into container stacks and/or from container stack tochassis/bombcart. This case is similar to the top-handler embodimentdescribed above, except that the image capture platform is a Rubber TireGantry (RTG) Crane, Rail Mounted Gantry (RMG) Crane, or other straddlecarrier equipment. In this case, all imaging and other data is sent tothe server over the Internet from the cameras or imaging equipment.

In the RTG/RMG OCR case, inventory is updated using the followingprocess: for an RTG/RMG container pull from UTR/Chassis/Bombcart/Draymanto the stack, the driver starts the RTG/RMG, in an embodiment, there aretwo cameras installed by each truck lane on the RTG/RMG (both sides,four cameras total) at the container height level designed to read thefront and back of the container. When a UTR bombcart, chassis, ordrayman drives under the truck lane of the RTG/RMG, the system imagesthe front and back container number of the container as it is on theUTR/Drayman. The system may also image the sides of the chassisidentification number if the container is on a chassis to indicatecontainer pull from a specific chassis. The PTIU installed on theRTG/RMG and tells container lock/pull from UTR/Drayman. When the RTG/RMGplaces the container on the stack, the system receives get a put messagefrom the PTIU, which tells exactly where the container was placed (tier,row, cell). The bay location is determined by two GPS's installed on theRTG/RMG, and the OCR system sends a message with the location of thecontainer.

In the case of a container PULL from the stack toUTR/Chassis/Bombcart/Drayman in the RTG/RMG system: the driver pullscontainer from stack location, the PTIU senses container pull event anddetermines row, bay, cell, tier. Based on the row, bay, cell, tier; thesystem locates container in stack and retrieves the container numberfrom the database. As the container is placed on a waiting UTR orChassis, container is imaged by the cameras once again as a doubleverification. The server updates with container movement (on UTR, inyard). The system may also image the sides of the chassis identificationnumber if the container is on a chassis to indicate container put to thewaiting chassis that receives the container.

In the case of a container reposition stack-to-stack in the RTG system,such as moving a container from a different location in a stack toanother location, the process proceeds as follows: the driver pullscontainer from stack location. The PTIU senses container pull event andthe system determines the row, bay, cell, tier. Based on the row, bay,cell, tier, the system locates the container in stack and retrieves thecontainer number from the database. The driver of the RTG puts thecontainer in a new stack position (it is not put on a UTR or Draymantruck so cameras are not used in this case). The PTIU records putlocation of container, and the database updates the location ofcontainer and sends message of the new container location.

The systems described above can be extended to or implemented within avariety of other container or object transport systems, such asship-based transport using quay cranes, bus-based transport usingforklifts, and other similar systems.

Mobile Inventory Vehicle System

In an embodiment, the images used for geotagging container images may begenerated by a mobile imaging unit, such as a car or truck upon whichappropriate imaging devices are mounted. In such a case, the vehicle maybe driven around a lot while an operator directly or remotely controlsthe camera to shoot photos or videos of the objects to be imaged. Thecamera on such a device may be synchronized to shoot pictures at regularintervals as the vehicle passes the objects. The vehicle may also beimplemented as an operator-less robot or other remote controlled mobiledevice. Such as system may be embodied as a mobile inventory vehicle(MIV) system in which a vehicle captures parked container and chassisIDs in a site every second (or other period of time) while driving ate.g., 5-10 mph (or faster) down parked rows of containers. The vehiclemay include an on-board server or computer that performs certain imagingand/or tagging or transmission operations. In an embodiment, theon-board vehicle computer may be configured to transmit the digitalimage data from the camera(s) to the remote server, for OCR processing,or it may be configured to perform the OCR processing locally. The OCRdata may need to be normalized according to numbering conventions, andthe system may be configured to accommodate both non-standard andstandard ISO container/chassis numbers. In this case, the system mayinclude an OCR engine that determines whether international or domestic(e.g., domestic U.S. containers and chassis) numbers are being used. Thevehicle may also include other computers that perform certain tasks,such as writing the data to memory (e.g., RAM disk, local hard drivestorage, optical drive storage, etc.), providing GPS or enhanced GPS, orother location information to the data for geotagging with the image.Certain calibration systems, such as parking spot cameras may be used toallow an operator to verify that the MIV cameras and GPS system areoperating properly and the container/chassis is parked in the space asdetermined by the GPS systems. Additional technology such as LED lamps,defrosters, and windshield wipers may be included to allow for thesystem to operate in a variety of lighting and weather conditions.

In an embodiment of the MIV system, two (or more) cameras may be used toprovide images of different views of the object (i.e. one camera forchassis, one for container imaging), different angles of the same viewof the object, images of the object using different focus or exposuresettings, or other cooperative images. Certain other technology may alsobe used to enhance the processing, such as the use of lasers or otherimaging to determine whether or not a parking spot is empty, whether ornot a container is present on top of a chassis (as in the case of abare-chassis), or the height of a container stack, among others.

In general, the camera-based OCR and geotagging system provides severaladvantages over present systems. For example, such as system can capturechassis number data much faster than manually writing or entering thenumbers; the process can be used on containers, chassis, license plates,anything with uniform characters to be imaged and transcribed to digitaltext; the system is safer in that it allows the user to remain in thesafe environment of a vehicle, reduces the requirement for users to beright next to or on containers, chassis, and other heavy equipment thatcan fall and injure the personnel in the yard; likewise, it reducesexposure to time required on-site in the dangerous terminal environmentand to harsh weather conditions, which can reduce the risk of injury dueto falling or slipping on wet or icy pavement. In this system,containers that cannot be read by the OCR system can still be enteredmanually with the audit client application, and processing can occur ina centralized office, which provides a safer and more environmentallyfriendly environment. The audit system can be run anywhere where thereis an Internet or network connection, and does not require the person tobe present at the site. The geotagging process allows the system toautomatically locate the container and/or chassis and display on a mapimage, and allows for locating the container and/or chassis withouthaving to enter a specific location, as well as for real time inventorycontrol of container and/or chassis. The system can image licenseplates, containers, chassis anywhere in the terminal where there aredefined parking zones of equipment with license plates, containers andchassis.

Although embodiments have been described with regard to the tracking ofcontainer chassis, this system is not limited to imaging chassis butcould also be used for license plates, container numbers, or any itemthat requires inventory control. For example, the MIV system may be usedfor reading license plates on parked cars in shopping malls, airports,etc, such as for security purposes or for finding stolen vehicles“dumped” at airports, malls, etc., or to locate lost vehicles, such aswhen someone texts their car license plate to a number and the locationis returned. The mobile phone embodiment can be used for trackingwarehouse (or similar confined environment) inventory, etc. A barcode orunique ID inside warehouse would be used to locate the equipment (as aGPS signal would not be in range inside a building for locationtracking). It could also be used to implement a low cost “gate OCR”system, in which an operator stands at a kiosk or gate as containers onchassis or vehicles enter a parking lot or port and images the licenseplate, chassis number, markings, damage (e.g., tire damage), orcontainer number, before they enter the yard. This provides acost-effective alternative to having a comprehensive installed gatesystem with cameras and wiring installed at the site.

It should be noted that the various functions disclosed herein may bedescribed using any number of combinations of hardware, firmware, and/oras data and/or instructions embodied in various machine-readable orcomputer-readable media, in terms of their behavioral, registertransfer, logic component, and/or other characteristics.Computer-readable media in which such formatted data and/or instructionsmay be embodied include, but are not limited to, physical(non-transitory), non-volatile storage media in various forms, such asoptical, magnetic or semiconductor storage media. Embodiments may alsoinclude cloud-based computing networks in which certain processing ordata storage tasks are provided through tenant-based client/servernetwork systems. In such an embodiment, image caching processes may beused to provide fast and safe access to particular users, such asinventory or site control managers who store image data on thecloud-based servers. Internet-based servers may also be used to reducethe necessity to use physical hardware or network interfaces at thesite. The network may be cellular based to reduce the need to utilizeterminal Wi-Fi networks. Hosting the server on the Internet allowscontainers to be tracked without any network at the imaging siterequired other than for an Internet connection.

Unless the context clearly requires otherwise, throughout thedescription and the claims, the words “comprise,” “comprising,” and thelike are to be construed in an inclusive sense as opposed to anexclusive or exhaustive sense; that is to say, in a sense of “including,but not limited to.” Words using the singular or plural number alsoinclude the plural or singular number respectively. Additionally, thewords “herein,” “hereunder,” “above,” “below,” and words of similarimport refer to this application as a whole and not to any particularportions of this application. When the word “or” is used in reference toa list of two or more items, that word covers all of the followinginterpretations of the word: any of the items in the list, all of theitems in the list and any combination of the items in the list.

While one or more implementations have been described by way of exampleand in terms of the specific embodiments, it is to be understood thatone or more implementations are not limited to the disclosedembodiments. To the contrary, it is intended to cover variousmodifications and similar arrangements as would be apparent to thoseskilled in the art. Therefore, the scope of the appended claims shouldbe accorded the broadest interpretation so as to encompass all suchmodifications and similar arrangements

What is claimed is:
 1. An apparatus, comprising: a handheld devicehaving a camera and a telecommunications interface and an imaging deviceconfigured to take a picture of an object, the picture containing anidentifier of the object; a global positioning system (GPS) componentassociated with the imaging device and configured to tag the image ofthe object with GPS location information of the object to generate atagged image; a communications interface configured to transmit thetagged image to a server computer remote from the handheld device overone of a telecommunications network protocol and an Internet Protocol(IP) network; and a processor of the server configured to performOptical Character Recognition (OCR) on the picture and to create anindicator code corresponding to the identifier of the object, whereinthe processor is further configured to create a processed resultcontaining the indicator code and the location to locate the objectwithin a site, wherein the GPS location information is derived by aknown area input of the location of the handheld device to locate theobject being imaged.
 2. The apparatus of claim 1, wherein the taggedimage comprises metadata including the identifier and information of achassis, a container or a handling equipment associated with the object.3. The apparatus of claim 1 wherein the GPS location information isembedded into the handheld device to provide the GPS locationinformation.
 4. The apparatus of claim 2, further comprising: imagingtires of the chassis; and bundling the tire and chassis image with theidentifier of the object so that an overall condition of the chassis canbe archived.
 5. The apparatus of claim 1, wherein the processor isconfigured to receive the tagged image through a web site, and whereinthe tagged image is displayed and processed in a web page displayinglocation information of the object in the site for onsite user review.6. The apparatus of claim 5 herein the web page comprises a plurality ofdisplay areas showing the location of the object relative to a map ofthe site and a description of the location relative to definedcoordinates of the site.
 7. The apparatus of claim 6 further comprisingan audit process providing display area configured to allow a user tocorrect an identifier processed by the OCR in the event of an errorcharacter recognition.
 8. The apparatus of claim 2 wherein the sitecomprises a container storage and transport terminal, and wherein theimaging device is mounted on at least one of: a front end loader,top-handler, a rubber tire gantry crane, and a rail-mounted gantrycrane.
 9. The apparatus of claim 2 wherein the imaging device is mountedat a gate location of the site for imaging container transport vehiclesas they enter or exit the site.
 10. A method comprising: receiving animage captured by an image capture device of an object within a definedsite, the image containing a unique identifier of the object;determining a geographical location of the object within the site;embedding the location of the object with the image of the object tocreate a tagged image; and transmitting the embedded location to aserver hosted on the Internet to be processed for identification andlocation for storage of the location information of the object ortransmission of the location information to a terminal operating system,wherein the GPS location information is derived by a known area input ofthe location of the handheld device to locate the object being imaged.11. The method of claim 10 further comprising: receiving the taggedimage from the image capture device over an Internet Protocol (IP)network; and performing Optical Character Recognition (OCR) of theidentifier in the image to create an indicator code including the imageand the location of the object.
 12. The method of claim 10 wherein theimaging device comprises a handheld smartphone comprising a camera and atelecommunications interface, and wherein the GPS component comprises acircuit resident in the smartphone.
 13. The method of claim 10 whereinthe object comprises one of: a chassis, a container or a handlingequipment associated with the object.